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CLAIMS 

1. A recording medium used for storing data, comprising: 
a digital stream generated by multiplexing a video 
5 stream and a graphics stream, wherein: 

the graphics stream includes a plurality of display 
sets each of Which is used for a graphics display; 

the display set includes a control segment and 
graphics data, the control segment including time 
10 information that designates an active period of the control 
segment in the display set on a reproduction time axis 
of the video stream; and 

when the active period of the control segment in the 
display set overlaps with an active period of a control 
15 segment in an immediately preceding display set, the time 
information designates the active period of the control 
segment in the display set to start at or after a time 
at which, during the active period of the control segment 
in the immediately preceding display set, transfer of 
20 graphics generated by decoding graphics data in the 
immediately preceding display set is completed. 

2. The recording medium of Claim 1, wherein: 

the active period of the control segment in the display 
25 set is from a decoding start time of the control segment 
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in the display set to a display start time of the graphics 
display which is composited based on the control segment 
in the display set. 

5 3. The recording medium of Claim 2, wherein: 

the control segment in the display set is provided 
at a beginning of the display set in a state of being 
contained within one packet; 

the time information includes a decoding time stamp 
10 and a presentation time stamp which are written in the 
packet; and 

the decoding time stamp shows the decoding start time , 
and the presentation time stamp shows the display start 
time. 

15 

4. The recording medium of Claim 1, wherein: 

the graphics data in the display set is made up of 
a sequence of pieces of graphics data a last one of which 
is immediately followed by an end code; and 
20 a time stamp associated with the end code shows a 

transfer completion time of the last piece of graphics 
data . 

5. The recording medium of Claim 4, wherein: 

25 the sequence of pieces of graphics data includes a 
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piece of referenced graphics data which is referenced by 

the control segment in the display set and a piece of 

non-referenced graphics data which is not referenced by 

the control segment in the display set; 
5 the piece of referenced graphics data and the piece 

of non-referenced graphics data are arranged in the display 

set in the stated order; 

the sequence of pieces of graphics data is to be 

sequentially decoded in the order of the arrangement; and 
10 the end code immediately follows the last piece of 

graphics data. 

6. The recording medium of Claim 1, wherein: • 

the display set further includes window information 

15 that specifies a position, a height, and a width of a window 
on a screen, the window being a rendering area in which 
the graphics display is to be composited based on the control 
segment in the display set before being overlaid on a moving 
picture represented by the video stream; and 

20 a time period from an end of the active period of 

the control segment in the immediately preceding display 
set to an end of the active period of the control segment 
in the display set is equal to a time period required for 
writing into the entire rendering area. 

25 
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7. The recording medium of Claim 1, 

wherein the active period of the control segment in 
the display set overlaps with the active period of the 
control segment in the immediately preceding display set 
5 if the display set and the immediately preceding display 
set belong to a same unit of memory management in the graphics 
stream; and 

if the display set and the immediately preceding 
display set belong to different units of memory management 
10 in the graphics stream, the time information designates 
the active period of the control segment in the display 
set to start at or after an end of the active period of 
the control segment in the immediately preceding display 
-set . 

15 

8. The recording medium of Claim 1, wherein: 

the graphics stream is a presentation graphics stream 
which is intended to synchronize with a moving picture 
represented by the video stream or an interactive graphics 
20 stream which is intended to produce an interactive display; 

the active period of the control segment in the display 
set overlaps with the active period of the control segment 
in the immediately preceding display set if the graphics 
stream is the presentation graphics stream; and 
25 if the graphics stream is the interactive graphics 
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stream, the time information designates the active period 
of the control segment in the display set to start at or 
after an end of the active period of the control segment 
in the immediately preceding display set. 

5 

9 . A reproduction apparatus for reproducing a digital 
stream generated by multiplexing a video stream and a 
graphics stream, comprising: 

a video decoder operable to decode the video stream 
10 to generate a moving picture; and 

a graphics decoder operable to decode the graphics 
stream to generate graphics, and overlay the graphics and 
the moving picture, wherein: 

the graphics stream includes a plurality of display 
15 sets each of which is used for a graphics display, the 
display set including a control segment and graphics data; 
and 

the graphics decoder performs pipeline processing, 
by starting processing the display set at or after a time 
20 at which, during an active period of a control segment 
in an immediately preceding display set, transfer of 
graphics generated by decoding graphics data in the 
immediately preceding display set to a buffer is completed. 

25 10. The reproduction apparatus of Claim 9, wherein: 
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the buffer is an object buffer for storing graphics 
generated by decoding; 

the graphics decoder includes : 

a processor operable to decode the graphics data in 
5 the display set to generate graphics, and transfer the . 
graphics to the object buffer; and 

a controller operable to read graphics from the ob j ect 
buffer based on the control segment in the display set, 
and overlay the read graphics and the moving picture; and 
10 in the pipeline processing, the processor transfers 

the graphics generated by decoding the graphics data in 
the display set to the object buffer, whilst simultaneously 
the controller reads graphics from the object buffer based 
on the control segment in the immediately preceding display 
15 set . 

11. The reproduction apparatus of Claim 10, wherein: 

the control segment in the display set is provided 
at a beginning of the display set; and 
20 the controller decodes the control segment, and, in 

accordance with a decoding result of the control segment, 
reads the graphics from the object buffer and displays 
the read graphics. 

25 12. The reproduction apparatus of Claim 11, wherein: 
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the control segment- in the display set is contained 
within one packet; and 

the controller starts decoding the control segment 
at a time shown by a decoding time stamp written in the 
5 packet, and starts displaying the graphics at a time shown 
by a presentation time stamp written in the packet. 

13. The reproduction apparatus of Claim 10, wherein: 

the graphics data in the display set is made up of 
10 a sequence of pieces of graphics data a last one of which 
is immediately followed by an end code; and 

a time stamp associated with the end code shows a 
transfer completion time of the last piece of graphics 
data . 

15 

14. The reproduction apparatus of Claim 13, wherein: 

the sequence of pieces of graphics data includes a 
piece of referenced graphics data which is referenced by 
the control segment in the display set and a piece of 
20 non-referenced graphics data which is not referenced by 
the control segment in the display set; 

the piece of referenced graphics data and the piece 
of non-referenced graphics data are arranged in the display 
set in the stated order; and 
25 the processor sequentially decodes the sequence of 
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pieces of graphics data in the order of the arrangement 
and transfers graphics generated by the decoding to the 
object buffer. 

■. 

5 15. The reproduction apparatus of Claim 14, wherein: 

the display set further includes window information 
that specifies a position, a height, and a width of a window 
on a screen, the window being a rendering area in which 
thegraphics display is to be composited based on the control. 
10 segment in the display set before being overlaid on the 
moving picture; 

the controller displays the graphics display 
composited based on the control segment in the display 
-set, a predetermined time period after displaying a 
15 graphics display composited based on the control segment 
in the immediately preceding display set; and 

the predetermined time period is equal to a time period 
required for writing into the entire rendering area. 

20 16 . ' The reproduction apparatus of Claim 9, wherein: 
the graphics decoder performs the pipeline 
processing if the display set and the immediately preceding 
display, set belong to a same unit of memory management 
in the graphics stream; and 

25 if the display set and the immediately preceding 
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display set belong to different units of memory management 
• in the graphics stream, the graphics- decoder starts 
processing the display set at or after a time at which 
display of the graphics display composited based on the 
6 control segment in the immediately preceding display set 
is started. 



17. The reproduction apparatus of Claim 9, wherein: 

the graphics stream is a presentation graphics stream 
10 which is intended to synchronize with the moving picture 
or an interactive graphics stream which is intended to 
produce an interactive display; 

the graphics decoder performs the pipeline 
-processing if the graphics stream is the presentation 
15 graphics stream; and 

if the graphics stream is the interactive graphics 
stream, the graphics decoder does not perform the pipeline 
processing . 



20 18. A method of recording onto a recording medium, 
comprising the steps of: 

generating application data; and 

recording the application data to the recording 
medium, wherein: 
25 the application data includes a digital stream 
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generated by multiplexing a video stream and a graphics 
stream; 

the graphics stream includes a plurality of display 
sets each of which is used for a graphics display; 

5 the display set includes a control segment and 

graphics data, the control segment including time 
information that designates an active period of the control 
segment in ' the display set on a reproduction time axis 
of the video stream; and 
10 when the active period of the control segment in the 

display set overlaps with an active period of a control 
segment in an immediately preceding display set, the time 
information designates the active period of the control 
segment in the display set to start at or after a time 

15 at which, during the active period of the control segment 
in the immediately preceding display set, transfer of 
graphics generated by decoding graphics data in the 
immediately preceding display set is completed. 

20 19 . A computer-readable program used for enabling a 
computer to reproduce a digital stream generated by 
multiplexing a video stream and a graphics stream, the 
program enabling the computer to perform the steps of : 
decoding the video stream to generate a moving 

25 picture; and 
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decoding the graphics stream to generate graphics, 
and overlaying the graphics and the moving picture, 
wherein : 

the graphics stream includes a plurality of display 
5 sets each of which is used for a graphics display, the 
display set including a control segment and graphics data; 
and 

the step of decoding the graphics stream perform 
pipeline processing, by starting processing the display 
10 set at or after a time at which, during an active period 
of a control segment in an immediately preceding display 
set, transfer of graphics generated by decoding graphics 
data in the immediately preceding display set to a buffer 
is completed. 

15 

20. A method of reproducing a digital stream generated 
by multiplexing a video stream and a graphics stream, 
comprising the steps of: 

decoding the video stream to generate a moving 
20 picture; and 

decoding the graphics stream to generate graphics, 
and overlaying the graphics and the moving picture, 
wherein : 

the graphics stream includes a plurality of display 
25 sets each of which is used for a graphics display, the 
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display set including a control segment and graphics data; 
and 

the step of decoding the graphics stream performs 
pipeline processing, by starting processing the display 
set at or after a time at which, during an active period 
of a control segment in an immediately preceding display 
set, transfer of graphics generated by decoding graphics 
data in the immediately preceding display set to a buffer 
is completed. 
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